Multicasting in Myrinet — A High-Speed, Wormhole-Routing Network

Wormhole routing networks are emerging as an effective solution for high bandwidth, low latency interconnects in distributed computing and cluster computing applications. An important example (in the local area environment) is the 640 Mb/s crossbar-based Myrinet. A key property of conventional LANs, which is valuable for many distributed applications, is transparent, reliable network level multicast. It is desirable to retain this property also in wormhole LANs. Unfortunately, efficient, reliable multicasting in wormhole LANs is problematic because of the potential for deadlocks. In this paper we address the problem of providing transparent, reliable, efficient network level multicasting in the wormhole LAN with special reference to Myrinet. We describe several protocols for achieving deadlock-free, reliable multicasting using restricted routing and fast buffer reservation techniques. Tradeoffs involving complexity and performance of various solutions are discussed, and are illustrated using simulation. Experimental results from a simple multicast implementation for Myrinet are presented.